Handwriting text summarization

ABSTRACT

The present disclosure relates to a computer-implemented method for handwriting-to-text-summarization, comprising obtaining, via a user interface of a system, a handwriting input representing a handwriting of a user of the system for handwriting-to-text-summarization, recognizing a text in the handwriting input, extracting at least one dynamic feature of the handwriting from the handwriting input, generating a text summary of the text, wherein generating the text summary is based on the text and on the at least one dynamic feature of the handwriting. The present disclosure also relates to a system for handwriting-to-text-summarization, comprising a user interface comprising a capturing subsystem configured to capture a handwriting of a user of the system, and wherein the system is configured to run the method for handwriting-to-text-summarization.

CROSS REFERENCE TO RELATED APPLICATIONS

This application claims priority from European patent application No.21305204.6, filed on Feb. 19, 2021, the contents of which are herebyincorporated herein in their entirety by this reference.

TECHNICAL FIELD

This specification relates to a computer-implemented method forhandwriting-to-text-summarization and to a system forhandwriting-to-text-summarization.

BACKGROUND

Automatic summarization is the process of shortening a textcomputationally, thereby generating a text summary that represents (themost) important or relevant information within the original content ofthe text. As an example, extraction-based summarization relies onclassifying the importance of sections of the text, usually at thesentence level. Once all sentences have been classified, the importantones are extracted and composed into the shorter text summary. Varioustechniques exist for successfully capturing the general importance of asentence to broader understanding of the text, ranging from simplermethods such as calculating the density of uncommon words to semanticanalysis of each sentence. In examples, abstraction-based summarizationinvolves natural language processing to build an internal semanticrepresentation of the original content of the text from which acondensed text summary is generated that is likely to be close to how ahuman might summarize the text.

Retaining dynamic features of handwriting of a user allows for assessingthe user based on non-textual (or non-verbal) contents of her or hiswritten input. In fact, as an example, it is known that dynamic featuressuch as measures of stroke distance, applied pressure and strokeduration extracted from the user writing by hand e.g. in a digital pensystem can be used to estimate the expertise/level of competence of theuser in the domain she or he is writing about. Along the same lines, itis possible to infer other attributes such as a level of confidence, anage of the user, and/or a given emotional state of the user from her orhis handwriting. While some of the dynamic features may easily beinterpreted by a human reader, pre-trained machine learning algorithmsare capable of also assessing subtler dynamic features.

In recent years, means of style transfer have evolved in the realm ofnatural language processing. Style transfer aims at changing a style ofa text while maintaining its linguistic meaning. As an example, a textwritten in an impolite style (e.g. an online review) can be converted toanother text conveying the same message but cast into a neutral orpolite style. Such transformations may rely on auto-encoders which canbe a class of neural networks (style transfer networks) configured tocreate in an encoding step a reduced and/or abstract representation ofthe text that is to be mapped in a decoding step to an output text. Incontrast to standard auto-encoders that are trained by providing theinput text also as the output text, auto-encoders for style transfer aretrained by providing style-transformed input texts as the output texts.

SUMMARY

According to a first aspect, there is provided a computer-implementedmethod for handwriting-to-text-summarization. The method comprisesobtaining, via a user interface of a system, a handwriting inputrepresenting a handwriting of a user of the system forhandwriting-to-text-summarization. The method further comprisesrecognizing a text in the handwriting input. The method furthercomprises extracting at least one dynamic feature of the handwritingfrom the handwriting input. The method further comprises generating atext summary of the text. Generating the text summary is based on thetext and on the at least one dynamic feature of the handwriting.

According to a second aspect, there is provided a system forhandwriting-to-text-summarization. The system comprises a user interfacecomprising a capturing subsystem configured to capture a handwriting ofa user of the system. The system is configured to run the method of thefirst aspect (or an embodiment thereof) forhandwriting-to-text-summarization.

Dependent embodiments of the aforementioned aspects are given in thedependent claims and explained in the following description, to whichthe reader should now refer.

Handwriting-to-text-summarization incorporates both importanceassessment of the text or portions (e.g. sentences/words) thereof andqualities of the text or the portions thereof indicated by dynamichandwriting features that reveal information about the user, such ase.g. the level of competence. In so doing, thehandwriting-to-text-summarization uses, retains and/or reflects suchinformation about the user, thereby yielding a text summary which ismore sensitive to subtle and/or invisible user input. Such may lead to abetter and more accurate text summary. In fact, as an example, in case asentence is rated to have been written with a low level of competence,it may not be included in the text summary.

Furthermore, handwriting-to-text-summarization may help to draw theattention of the user as well as of a third party (e.g. a parent and/ora teacher) to particular education needs, such as identifying gaps inknowledge and/or understanding. In fact, generating the text summary ofthe text may depend on settings that the user or the third party (evenafter completion of the user's handwriting) may adjust. As an example,such settings can be used to generate a text summary that deliberatelyfeatures and/or highlights portions of the text that correspond to lowlevel of competence and/or low level of confidence of the user. Such mayallow a teacher to efficiently identify the aforementioned gaps. Hence,the system for handwriting-to-text-summarization can be used as a meansin education and/or teaching.

In addition, a set of routines may run on the generated text summary toaccomplish goals such as style transfer, font modification, and/orannotation. In fact, the language can be adjusted to match psychologicalor educational information retrieved from handwriting dynamics, thusmaking this information more obvious to readers (i.e. the user or thethird party). Such routines/information may again be used to identifygaps in knowledge and/or understanding. Along the same lines, the textsummary may be automatically annotated according to information fromhandwriting dynamics, and thus not requiring teachers or students tohave to do in-depth analysis of their knowledge gaps. Annotations may beinteractive, e.g. in terms of hyperlinks encoding a request to a searchengine on the web.

In general, note taking while following an event such as e.g., a lectureor a business meeting is typically a lossy process in that given thesomewhat limited multitasking capabilities of a human brain there can bea trade-off between jottings things down and digesting newly incomingcontent or interacting in the event. As an example, that may lead to asuboptimal timing of writing versus listening/talking which may hamperfuture use of the notes rendering them less telling and thus lessbeneficial to the user in the long run. However, text summaries frommultiple users attending the same event (e.g. the lecture or thebusiness meeting) and each jotting down notes in a system forhandwriting-to-text-summarization may be collated based on theinformation about the users such as e.g. their level ofcompetence/domain expertise, so that the resulting collated text summaryrepresents an expertise level which can be higher than any one of theindividual text summaries of the multiple users. In so doing, collatedhandwriting-to-text-summarization (e.g. via a network of systems forhandwriting-to-text-summarization) can be seen as a means forcollective/crowd intelligence improving the text summary.

FIGURES DESCRIPTION

FIG. 1 schematically illustrates a computer-implemented method accordingto the first aspect (or an embodiment thereof) forhandwriting-to-text-summarization.

FIG. 2 schematically illustrates a system according to the second aspect(or an embodiment thereof) for handwriting-to-text-summarization.

FIG. 3 schematically illustrates an embodiment of thecomputer-implemented method according to the first aspect.

FIG. 4 illustrates an example for handwriting-to-text-summarization.

FIG. 5a illustrates an example for handwriting-to-text-summarizationwith style transfer.

FIG. 5b illustrates an example for handwriting-to-text-summarizationwith font modification.

FIG. 5c illustrates an example for handwriting-to-text-summarizationwith annotation.

FIG. 5d illustrates an example for handwriting-to-text-summarizationafter collation.

FIG. 6 shows an example machine learning training flow chart.

FIG. 7 illustrates an implementation of a general computer system thatmay execute techniques presented herein.

DETAILED DESCRIPTION

The method 100 of the first aspect (or an embodiment thereof) and thesystem 200 of the second aspect (or an embodiment thereof) aim atproviding functionality for handwriting-to-text-summarization. As anexample, as shown in FIG. 4, a user of the system 200 may use a writingutensil 221 such as e.g. a digital pen or a smart pen to capture ahandwriting input 10 that is to be summarized in terms of a text summary40.

The computer-implemented method 100 forhandwriting-to-text-summarization comprises obtaining 110, via a userinterface 210 of a system 200, a handwriting input 10 representing ahandwriting of a user of the system 200 forhandwriting-to-text-summarization. The method further comprisesrecognizing 120 a text 20 in the handwriting input 10. The methodfurther comprises extracting 130 at least one dynamic feature 30 of thehandwriting from the handwriting input 10. The method further comprisesgenerating 140 a text summary 40 of the text 20. Generating 140 the textsummary 40 is based on the text 20 and on the at least one dynamicfeature 30 of the handwriting. The computer-implemented method 100 isschematically illustrated in FIG. 1 and an embodiment thereof in FIG. 3.The text 20 may be given in terms of a character encoding, e.g. in plaintext ASCII.

The one or more dynamic features 30 may comprise or be an averagewriting pressure, an average stroke length, and/or an average strokeduration, wherein averaging is over the text 20 or portions thereof. Asan example, a dynamic feature 30 is an average writing pressure.Alternatively, or in addition, the dynamic feature 30 or another dynamicfeature 30 is an average stroke length. Alternatively, or in addition,the dynamic feature 30 or yet another dynamic feature 30 is an averagestroke duration. The text 20 can be semantically and/or linguisticallyinterpretable with respect to at least one communication language.

The handwriting input 10 may comprise a first set of data representingthe text. As an example, the first set of data may comprise at least onetime series (or at least one vector) of stroke data captured by acapturing system 220 of the system 200 forhandwriting-to-text-summarization. The first set of data may representthe text 20, if the text 20 written by the user of the system 200 interms of his or her handwriting can be reproduced from the first set ofdata, e.g. from the one or more time series (or vectors) of stroke data.Furthermore, the handwriting input 10 may comprise a second set of datarepresenting properties of the handwriting that indicate informationabout the user as handwriting progresses. As an example, the second setof data representing properties of the handwriting may be at least onetime series (or at least one vector) of pen pressure or pressure on atouchpad. The information about the user may relate to an emotionalstate, an age, a level of confidence, and/or a level of competence (viz.domain expertise) of the user. The first set of data and the second setof data overlap or are identical.

Recognizing 120 the text 20 in the handwriting input 10 may compriseapplying the handwriting input 10 (e.g. the first set of data) to a textpre-processing algorithm configured to recognize the text 20 representedby the handwriting input 10. Furthermore, the text pre-processingalgorithm can be further configured to segment 121 the text 20 into oneor more portions. As an example, such a segmentation may follow from ananalysis of punctuation marks (periods, colons, semi-colons, commas)and/or indentation and/or bullet points. The order of the portions canbe maintained. In examples, portions can be enumerated in their originalorder so that their original order can be reproduced any time. In fact,the one or more portions can be enumerated as they appear in the text.Alternatively, or in addition, the one or more portions can betimestamped. As an example, the one or more portions (of the text 20)may be sentences, clauses, and/or phrases (of the text 20).

The text pre-processing algorithm may comprise or be a machine-learningalgorithm pre-trained for handwriting-to-text recognition and/or textsegmentation. The text pre-processing algorithm may be configured tosegment the first set of data into individual character vectors andapply a pre-determined vector-to-character mapping to output the text.

The text pre-processing algorithm may be further configured to store thetext 20 in a database. Furthermore, the text pre-processing algorithmmay be further configured to store the one or more portions of text 20(and information about their order, e.g. their enumeration and/ortimestamps) in the database. The database may or may not form part ofthe system 200. In the latter case, the database can be hosted on aserver (e.g. in a cloud). Saving the text 20 and/or the portions thereof(and information about their order) in the database allows for reuse ofthe text 20. As an example, such can be useful when collating 180 textsummaries 40 of various users.

Extracting 130 the at least one dynamic feature 30 of the handwritingfrom the handwriting input 10 may comprise applying the handwritinginput 10 to a handwriting dynamics algorithm configured to extract theat least one dynamic feature 30 from the handwriting input 10. Thehandwriting dynamics algorithm may be further configured to compute atleast one dynamic feature 30 for each portion of the text. Thehandwriting dynamics algorithm may be further configured to compute theaverage writing pressure, the average stroke length, and/or the averagestroke duration, wherein averaging is over the text 20 or portions (e.g.sentences) thereof, thereby producing the one or more dynamic features30 for the text 20 or for each portion thereof. Computing the averagewriting pressure, the average stroke length, and/or the average strokeduration may be based on the second set of data.

The handwriting dynamics algorithm can be further configured to storethe one or more dynamic features 30 for the text 20 in the database. Thehandwriting dynamics algorithm may be further configured to store theone or more dynamic features 30 for each portion of the text 20 (andinformation about the order of the portions, e.g. their enumeration ortimestamps) in the database.

As an example, the text 20 and the one or more dynamic features 30 canbe stored in terms of a data structure of text (and/or portions of text)and corresponding one or more dynamic features.

Generating 140 the text summary 40 of the text 20 based on the text 20and on the at least one dynamic feature 30 of the handwriting maycomprise applying the text 20 and the one or more dynamic features 30 toa text summarization algorithm configured to generate the text summary40 of the text 20. The text summarization algorithm may comprise or be anatural language processing machine-learning algorithm pre-trained forautomatic summarization such as e.g. extraction-based orabstraction-based text summarization. Extraction-based orabstraction-based text summarization can be extended in that, in thetraining of the natural language processing machine-learning algorithm,the one or more dynamic features 30 can be provided as additional inputdata (in addition to the text 20 or its portions).

As an example, as in FIG. 3, the text summarization algorithm maycomprise applying 141 the one or more portions of the text 20 to animportance classifier configured to classify each portion of the text 20in terms of at least two classes indicating different levels ofimportance. In one embodiment, the classifier may take data from aplurality of sources, wherein data from the plurality of sources mayhave different weights. In one example embodiment, if the classifier isnot confident in terms of the content of the text, the classifier mayput more emphasis on the handwriting features. As an example, the atleast two classes may be labelled “important” and “not important”. Inexamples, in case of three classes, the classes may be labelled “veryimportant”, “important”, and “not important”. Importance may indicatethat a portion shall be included or contribute to the text summary 40.The importance classifier may be a natural language processingmachine-learning algorithm pre-trained for extraction-based textsummarization. In examples, instead of the importance classifier aregression algorithm can be used to output an importance value (e.g. areal number in the interval [0, 1]) that, if needed, can be classifiedupon discretization of the importance value.

As in FIG. 3, the text summarization algorithm may comprise applying 142the one or more dynamic features 30 for each portion of the text 20 toat least one quality classifier configured to classify the one or moredynamic features 30 for each portion of the text 20 in terms of at leasttwo classes relating to information about the user.

In examples, qualities can be properties of the written language whichindicate certain meanings, styles or tones to a user. For example, aquality may include how expert a writer seems, how authoritative theyare, how child-like or old they are, their emotional state, or other.Qualities may be indicated in any aspect of writing or handwriting,including the graphical form of the handwriting, the properties of thedynamic motions used to create the handwriting, the word choice,language construction, or other. Some qualities may be easily identifiedby humans, and some qualities may only be easily recognizedalgorithmically. This is largely dependent on which aspects of thewriting the quality indicate. As an example, simplistic word use can beeasily recognized by a human as being “child-like”, but subtle changesin applied pressure may only indicate domain expertise level to analgorithm.

The at least one quality classifier may correspond to one or more of theemotional state, the age, a level of confidence, and the level ofcompetence of the user. As an example, the at least one qualityclassifier may correspond to the emotional state of the user.Alternatively, or in addition, the at least one quality classifier oranother quality classifier may correspond to the age of the user.Alternatively, or in addition, the at least one quality classifier oranother quality classifier may correspond to the level of confidence ofthe user. Alternatively, or in addition, the at least one qualityclassifier or another quality classifier may correspond to the level ofcompetence of the user. Alternatively, or in addition, the at least onequality classifier may output a vector (e.g. a two-dimensional vector, athree-dimensional vector, or a four-dimensional vector) for any two orthree combinations of the emotional state, the age, the level ofconfidence, and the level of competence of the user, or for thecombination of the emotional state, the age, the level of confidence,and the level of competence of the user.

In an embodiment, the at least one quality classifier may correspond tothe level of competence of the user. As an example, classifying thelevel of competence of the user may be in terms of two or three classes(e.g. “expert” and “novice” or e.g. “expert”, “normal”, and “novice”).As an example, the level of competence can be decisive when it comes todeciding which portions of the text 20 is relevant for the text summary40. In examples, the level of competence may be decisive when collating(180) text summaries of several users and deciding that a portion of auser shall contribute to the (collated) text summary 40. The one or morequality classifiers may be a machine-learning algorithm in a pre-trainedstate (i.e. after training on training data). As an example, the one ormore quality classifiers may be trained on results of specificuser-groups relevant to the use case, e.g. children, students orlearners. For example, the quality classifier corresponding to the levelof competence may have been trained on examples of “expert”, “normal”and “novice”. In this context, the class “expert” may refer tohandwriting input 10 (and the one or more dynamic features extractedtherefrom) produced by children or students known to be very competentin that area. The class “normal” may refer to handwriting input 10 (andthe one or more dynamic features extracted therefrom) produced bychildren or students with only limited exposure to that area. The class“novice” may refer to handwriting input 10 (and the one or more dynamicfeatures extracted therefrom) produced by children or students workingon a totally new area.

In an embodiment, as illustrated in FIG. 3, the text summarizationalgorithm may comprise applying 143 a portion ranking algorithmconfigured to assign a ranking value to each portion of the text 20based on the corresponding classification results 41 of the importanceclassifier and on the corresponding classification results 42 of the atleast one quality classifier. Assigning a ranking value SR of eachportion of the text 20 based on the corresponding classification results41 of the importance classifier and on the corresponding classificationresults 42 of the at least one quality classifier may comprise computinga linear combination

SR=aIC+(b ₁ Q ₁ C ₁ + . . . +b _(N) QC _(N))/N

of a numeric result IC of the importance classifier and a normalizedlinear combination of numeric results QC₁, QC₂, . . . , QC_(N) of the Nquality classifiers.

The numeric results of the importance classifier and of the N qualityclassifiers can be pre-determined values corresponding to the respectiveclasses of the importance classifier or the N quality classifiers,respectively. As an example, the pre-determined values can beuser-defined, i.e. they can be adjusted in a settings menu of the userinterface 210 of the system 200. Some of them can be chosen to be zero(e.g. to discard normal level of competence). As an example, the numericresult of the class “expert” can be one and the numeric results of theclasses “normal” or “novice” can be zero. In examples, the numericresults of the classes “expert” or “novice” can be one and the numericresult of the class “normal” can be zero. In examples, in case ofreplacing classifiers by regressors the numeric results can be theoutput values of the regressors.

Coefficients, e.g. a, of the linear combination and/or coefficients b₁,. . . , b_(N) of the normalized linear combination can be pre-determinedweights. Again, the pre-determined weights can also be user-defined,i.e. they can be adjustable in the settings menu of the user interface210. They can also be set to zero (e.g. to eliminate or suppress vaguestatements from non-experts).

The (one or more) portions of the text 20 with corresponding rankingvalues above a predetermined threshold value can be concatenated (e.g.in the order the one or more portions appear in the text 20), therebygenerating 140 the text summary 40. The predetermined threshold valuemay also be user-defined, i.e. it can be adjusted in the settings menuof the user interface 210. A lower threshold may result in a lesscompressed summarization. The right order of the one or more portions(e.g. when queried from the database) can be restored from theenumeration or the timestamps.

In an embodiment, the method 100 may further comprise applying 150 thetext summary 40 to a style transfer algorithm configured to modify thetext summary 40 so as to reflect the information about the user. Thestyle transfer algorithm may apply at least one style transfer network.In one embodiment, different areas of the text may have different fonts,e.g., italics, bold, underlined, etc., to notify the users on theimportance of each specific area of their notes.

Modifying the text summary 40 may comprise selecting for each portion ofthe text summary 40 one or more style transfer networks (also selectingtheir order) based on the one or more classification results 42 of theone or more quality classifiers (e.g. queried from the database), andmodifying each portion of the text summary 40 by applying thecorresponding one or more transfer networks, thereby modifying the textsummary 40. The selection of the one or more style transfer networks maybe influenced by user settings. As an example, style transfer networkassociated to the level of competence may apply the linguistic style ofan expert to the input text, such that the output appears to have beenwritten by an expert (e.g. confident word choice, no hedge words). Theneural network itself may have been trained on examples of expertwritings in different domains (levels of competence), such that the“expert” language is not necessarily domain specific. An example of atext summary 40 after style transformation is shown in FIG. 5 a.

In examples, the at least one style transfer network may be anauto-encoder neural network in a pre-trained state (i.e. after trainingon training data).

In an embodiment, the method 100 may further comprise applying 160 thetext summary 40 to a font modification algorithm configured to changethe font of at least one portion of the text summary 40 based on theinformation about the user. In one example embodiment, the fontmodification algorithm may change the font of at least one portion ofthe text based, at least in part, on emotional state, age, level ofconfidence, a level of competence, or a combination thereof of the user.Changing the font of the at least one portion of the text summary 40based on the information about the user may comprise querying for eachportion of the text summary 40 a font based on the one or moreclassification results 42 of the one or more quality classifiersmatching the font labels of the font, and formatting each portion in thecorresponding font, thereby modifying the text summary 40. At least onefont may be queried from a font database or from the database. Changingthe font depending e.g. on the level of competence of the user of thesystem 200 can be seen as a feedback means to inform the user ofportions of the text 20 that are deemed to be less certain. It may alsobe conducive to enhancing the perceptibility of the text summary 40 tothe user (also when reviewing the text summary 40 at a later time). Inso doing, such feedback may contribute to a better understanding of thetext 20. An example of a text summary 40 after font modification isshown in FIG. 5 b.

In an embodiment, the method 100 may further comprise applying 170 thetext summary 40 to an annotation algorithm configured to add at leastone annotation to the text summary 40 based on the information about theuser. Adding the at least one annotation to the text summary 40 based onthe information about the user may comprise querying, for each portionof the text summary 40, an annotation database so as to find a matchingannotation to the portion (or parts thereof such as e.g. a selection ofone or more words of the portion) and to the one or more classificationresults 42 of the one or more quality classifiers, and evaluating apredetermined trigger condition, and adding the matching annotation tothe portion, if a matching annotation has been found and thepredetermined trigger condition is satisfied, or adding a universalannotation to the portion, if a matching annotation has not been foundand the predetermined trigger condition is satisfied, thereby modifyingthe text summary 40.

Each annotation may have a specific level of competence trigger, i.e.“expert”, “normal”, “novice” which can be used in the assignment of oneor more annotations to a specific portion of the text 20. A matchingannotation, such as e.g. “Dolphins are mammals and therefore need tobreathe air”, may be pre-defined by a teacher, parent, or other. Thesemay have specific pre-defined trigger words or combinations of words,such as “dolphin”, “mammal”, “breathing”.

The matching annotation or the universal annotation may result from aqueried annotation template after parametrization based on the portionand/or the one or more classification results 42 of the one or morequality classifiers. As an example, the universal annotation can be aninteractive hyperlink (e.g. via the user interface 210 of the system200) to a search engine. In fact, a parametrization can consist inadding the interactive hyperlink to a phrase such as e.g. “more helphere: (hyperlink)”. An example of a text summary 40 with annotation(s)is shown in FIG. 5 c.

In an embodiment, the method 100 may further comprise applying 180 thetext summary 40 to a collation algorithm configured to compare the textsummary 40 to one or more further text summaries corresponding to one ormore further users of one or more further systems 200 forhandwriting-to-text-summarization (e.g. a network of systems 200 forhandwriting-to-text-summarization), and to sample a collated textsummary 40 based on the information about the user and on informationabout the one or more further users, thereby modifying the text summary40. In one embodiment, the data from the plurality of users taking notesat the same time are implemented to train the collation algorithm. Thecollation algorithm may then train the system to evaluate the notes ofthe plurality of users to provide feedback. For example initially, thesystem could provide feedback in a general way, but as more and morepeople from the same classroom are actually taking notes (e.g., aplurality of students are taking notes for a particular subject), thesystem may process all these notes to generate an average value toprovide feedback to the students of similar classes. Collating textsummaries 40 (the text summary and the further text summaries) can beused when note taking is for the same event (e.g. a business meeting, alecture). Comparing the text summary 40 to one or more further textsummaries may comprise identifying for each portion (or for each of asubset of portions) of the text summary 40 a list of correspondingfurther portions in the one or more further text summaries.Alternatively, or in addition, text summaries 40 can be compared at theword and/or phrase-level.

Sampling a collated text summary 40 based on the information about theuser and on the information about the one or more further users maycomprise selecting a portion or a further portion for each list based onthe information about the user and on the information about the one ormore further users. The information about the user and the informationabout the one or more further users may correspond to the level ofcompetence of the user or the further users, respectively. Level ofcompetence can be decisive when deciding which portion shall be includedin the collated text summary. As an example, in case where multiplevalid or equivalent portions have an “expert” classification, one ofthem may be chosen at random. In examples, in case of a portion havingno equivalents in the further text summaries, the portion may beimmediately selected by default. In some cases, for example, if ateacher wants to understand where students are commonly missingknowledge, one or more portions with an “novice” classification may beselected instead. An example of a text summary 40 after collation isshown in FIG. 5 d.

The method 100 may further comprise displaying 190 the text summary 40via a graphical output 230 of the user interface 210 of the system 200for handwriting-to-text-summarization. User settings can be set via theuser interface 210 of the system 200.

The system 200 for handwriting-to-text-summarization, may comprise auser interface 210 comprising a capturing subsystem 220 configured tocapture a handwriting of a user of the system 200, and wherein thesystem 200 is configured to run the method 100 of one of the precedingclaims. Such a system is schematically illustrated in FIG. 2.

The capturing subsystem 220 may comprise a touchpad, a touch screen, agraphics tablet, or a digital tablet. Capturing the handwriting of theuser of the system 200 may comprise capturing handwriting input 10representing text 20 written by hand or writing utensil 221 by the useron the touchpad, the touch screen, the graphics tablet, or the digitaltablet. As an example, a touch screen or digital tablet may be capableof capturing stroke vectors, which may contain information such asstroke pressure, stroke duration and/or stroke distance.

The capturing subsystem 220 may comprise or be integrated in a writingutensil 221. The writing utensil 221 is a ballpoint pen, a fountain pen,a felt-tip pen, a brush, a pencil. Alternatively, or in addition, thewriting utensil 221 can be a digital pen or a smart pen. As in FIG. 4,the writing utensil 221 may comprise a motion sensor module 222comprising one or more of one or more accelerometers, one or moregyroscopes, one or more magnetometers, and one or more force or pressuresensors. The motion sensor module 222 may be configured to capturehandwriting motion data comprising one or more of pressure, speed,acceleration, and direction of the writing utensil 221 when being usedby the user to write text 20 by hand. Capturing the handwriting of theuser of the system 200 comprises capturing handwriting input 10comprising handwriting motion data of the writing utensil 221 when beingused by the user to write text 20 by hand.

The user interface 210 may comprise a graphical output 230. Thegraphical output 230 (e.g. a touch screen) may allow for userinteraction.

The system 200 may comprise at least one database. Alternatively, or inaddition, the system 200 may access a database in a cloud (via acommunication interface). The system 200 may comprise a (at least one)communication interface 240 to couple to one or more systems 200 forhandwriting-to-text-summarization. The communication interface 240 maycomprise one or more of a network, internet, a local area network, awireless local area network, a broadband cellular network, and/or awired network. In examples, the system may couple to one or more systems200 via a server hosted in a cloud. As an example, such networkconnectivity can be used when collating the text summary 40 to thefurther text summaries.

One or more implementations disclosed herein include and/or may beimplemented using a machine learning model. For example, one or more ofthe text pre-processing algorithm, machine-learning algorithm,handwriting dynamics algorithm, text summarization algorithm, regressionalgorithm, portion ranking algorithm, style transfer algorithm, fontmodification algorithm, annotation algorithm, and/or collation algorithmmay be implemented using a machine learning model and/or may be used totrain a machine learning model. A given machine learning model may betrained using the data flow 610 of FIG. 6. Training data 612 may includeone or more of stage inputs 614 and known outcomes 618 related to amachine learning model to be trained. The stage inputs 614 may be fromany applicable source including text, visual representations, data,values, comparisons, stage outputs (e.g., one or more outputs from astep from FIGS. 1, 2, and/or 3). The known outcomes 618 may be includedfor machine learning models generated based on supervised orsemi-supervised training. An unsupervised machine learning model may notbe trained using known outcomes 618. Known outcomes 618 may includeknown or desired outputs for future inputs similar to or in the samecategory as stage inputs 614 that do not have corresponding knownoutputs.

The training data 612 and a training algorithm 620 (e.g., one or more ofthe text pre-processing algorithm, machine-learning algorithm,handwriting dynamics algorithm, text summarization algorithm, regressionalgorithm, portion ranking algorithm, style transfer algorithm, fontmodification algorithm, annotation algorithm, and/or collation algorithmimplemented using a machine learning model and/or may be used to train amachine learning model) may be provided to a training component 630 thatmay apply the training data 612 to the training algorithm 620 togenerate a machine learning model. According to an implementation, thetraining component 630 may be provided comparison results 616 thatcompare a previous output of the corresponding machine learning model toapply the previous result to re-train the machine learning model. Thecomparison results 616 may be used by the training component 630 toupdate the corresponding machine learning model. The training algorithm620 may utilize machine learning networks and/or models including, butnot limited to a deep learning network such as Deep Neural Networks(DNN), Convolutional Neural Networks (CNN), Fully Convolutional Networks(FCN) and Recurrent Neural Networks (RCN), probabilistic models such asBayesian Networks and Graphical Models, and/or discriminative modelssuch as Decision Forests and maximum margin methods, or the like.

A machine learning model used herein may be trained and/or used byadjusting one or more weights and/or one or more layers of the machinelearning model. For example, during training, a given weight may beadjusted (e.g., increased, decreased, removed) based on training data orinput data. Similarly, a layer may be updated, added, or removed basedon training data/and or input data. The resulting outputs may beadjusted based on the adjusted weights and/or layers.

In general, any process or operation discussed in this disclosure thatis understood to be computer-implementable, such as the processillustrated in FIGS. 1, 2, and/or 3 may be performed by one or moreprocessors of a computer system as described above. A process or processstep performed by one or more processors may also be referred to as anoperation. The one or more processors may be configured to perform suchprocesses by having access to instructions (e.g., software orcomputer-readable code) that, when executed by the one or moreprocessors, cause the one or more processors to perform the processes.The instructions may be stored in a memory of the computer system. Aprocessor may be a central processing unit (CPU), a graphics processingunit (GPU), or any suitable types of processing unit.

A computer system, such as a system or device implementing a process oroperation in the examples above, may include one or more computingdevices. One or more processors of a computer system may be included ina single computing device or distributed among a plurality of computingdevices. One or more processors of a computer system may be connected toa data storage device. A memory of the computer system may include therespective memory of each computing device of the plurality of computingdevices.

In various embodiments, one or more portions of method 100 and system200 may be implemented in, for instance, a chip set including aprocessor and a memory as shown in FIG. 7. FIG. 7 illustrates animplementation of a general computer system that may execute techniquespresented herein. The computer system 700 can include a set ofinstructions that can be executed to cause the computer system 700 toperform any one or more of the methods, system, or computer basedfunctions disclosed herein. The computer system 700 may operate as astandalone device or may be connected, e.g., using a network, to othercomputer systems or peripheral devices.

Unless specifically stated otherwise, as apparent from the followingdiscussions, it is appreciated that throughout the specification,discussions utilizing terms such as “processing,” “computing,”“determining”, or the like, refer to the action and/or processes of acomputer or computing system, or similar electronic computing device,that manipulate and/or transform data represented as physical, such aselectronic, quantities into other data similarly represented as physicalquantities.

In a similar manner, the term “processor” may refer to any device orportion of a device that processes electronic data, e.g., from registersand/or memory to transform that electronic data into other electronicdata that, e.g., may be stored in registers and/or memory. A “computer,”a “computing machine,” a “computing platform,” a “computing device,” ora “server” may include one or more processors.

In a networked deployment, the computer system 700 may operate in thecapacity of a server or as a client user computer in a server-clientuser network environment, or as a peer computer system in a peer-to-peer(or distributed) network environment. The computer system 700 can alsobe implemented as or incorporated into various devices, such as apersonal computer (PC), a tablet PC, a personal digital assistant (PDA),a mobile device, a palmtop computer, a laptop computer, a desktopcomputer, a communications device, a wireless telephone, a land-linetelephone, a control system, a camera, a scanner, a facsimile machine, apersonal trusted device, a web appliance, a network router, switch orbridge, or any other machine capable of executing a set of instructions(sequential or otherwise) that specify actions to be taken by thatmachine. In a particular implementation, the computer system 700 can beimplemented using electronic devices that provide voice, video, or datacommunication. Further, while a computer system 700 is illustrated as asingle system, the term “system” shall also be taken to include anycollection of systems or sub-systems that individually or jointlyexecute a set, or multiple sets, of instructions to perform one or morecomputer functions.

As illustrated in FIG. 7, the computer system 700 may include aprocessor 702, e.g., a central processing unit (CPU), a graphicsprocessing unit (GPU), or both. The processor 702 may be a component ina variety of systems. For example, the processor 702 may be part of astandard personal computer or a workstation. The processor 702 may beone or more general processors, digital signal processors, applicationspecific integrated circuits, field programmable gate arrays, servers,networks, digital circuits, analog circuits, combinations thereof, orother now known or later developed devices for analyzing and processingdata. The processor 702 may implement a software program, such as codegenerated manually (i.e., programmed).

The computer system 700 may include a memory 704 that can communicatevia a bus 708. The memory 704 may be a main memory, a static memory, ora dynamic memory. The memory 704 may include, but is not limited tocomputer readable storage media such as various types of volatile andnon-volatile storage media, including but not limited to random accessmemory, read-only memory, programmable read-only memory, electricallyprogrammable read-only memory, electrically erasable read-only memory,flash memory, magnetic tape or disk, optical media and the like. In oneimplementation, the memory 704 includes a cache or random-access memoryfor the processor 702. In alternative implementations, the memory 704 isseparate from the processor 702, such as a cache memory of a processor,the system memory, or other memory. The memory 704 may be an externalstorage device or database for storing data. Examples include a harddrive, compact disc (“CD”), digital video disc (“DVD”), memory card,memory stick, floppy disc, universal serial bus (“USB”) memory device,or any other device operative to store data. The memory 704 is operableto store instructions executable by the processor 702. The functions,acts or tasks illustrated in the figures or described herein may beperformed by the processor 702 executing the instructions stored in thememory 704. The functions, acts or tasks are independent of theparticular type of instructions set, storage media, processor orprocessing strategy and may be performed by software, hardware,integrated circuits, firm-ware, micro-code and the like, operating aloneor in combination. Likewise, processing strategies may includemultiprocessing, multitasking, parallel processing and the like.

As shown, the computer system 700 may further include a display 710,such as a liquid crystal display (LCD), an organic light emitting diode(OLED), a flat panel display, a solid-state display, a cathode ray tube(CRT), a projector, a printer or other now known or later developeddisplay device for outputting determined information. The display 710may act as an interface for the user to see the functioning of theprocessor 702, or specifically as an interface with the software storedin the memory 704 or in the drive unit 706.

Additionally or alternatively, the computer system 700 may include aninput/output device 712 configured to allow a user to interact with anyof the components of computer system 700. The input/output device 712may be a number pad, a keyboard, or a cursor control device, such as amouse, or a joystick, touch screen display, remote control, or any otherdevice operative to interact with the computer system 700.

The computer system 700 may also or alternatively include drive unit 706implemented as a disk or optical drive. The drive unit 706 may include acomputer-readable medium 722 in which one or more sets of instructions724, e.g. software, can be embedded. Further, instructions 724 mayembody one or more of the methods or logic as described herein. Theinstructions 724 may reside completely or partially within the memory704 and/or within the processor 702 during execution by the computersystem 700. The memory 704 and the processor 702 also may includecomputer-readable media as discussed above.

In some systems, a computer-readable medium 722 includes instructions724 or receives and executes instructions 724 responsive to a propagatedsignal so that a device connected to a network 770 can communicatevoice, video, audio, images, or any other data over the network 770.Further, the instructions 724 may be transmitted or received over thenetwork 770 via a communication port or interface 720, and/or using abus 708. The communication port or interface 720 may be a part of theprocessor 702 or may be a separate component. The communication port orinterface 720 may be created in software or may be a physical connectionin hardware. The communication port or interface 720 may be configuredto connect with a network 770, external media, the display 710, or anyother components in computer system 700, or combinations thereof. Theconnection with the network 770 may be a physical connection, such as awired Ethernet connection or may be established wirelessly as discussedbelow. Likewise, the additional connections with other components of thecomputer system 700 may be physical connections or may be establishedwirelessly. The network 770 may alternatively be directly connected to abus 708.

While the computer-readable medium 722 is shown to be a single medium,the term “computer-readable medium” may include a single medium ormultiple media, such as a centralized or distributed database, and/orassociated caches and servers that store one or more sets ofinstructions. The term “computer-readable medium” may also include anymedium that is capable of storing, encoding, or carrying a set ofinstructions for execution by a processor or that cause a computersystem to perform any one or more of the methods or operations disclosedherein. The computer-readable medium 722 may be non-transitory, and maybe tangible.

The computer-readable medium 722 can include a solid-state memory suchas a memory card or other package that houses one or more non-volatileread-only memories. The computer-readable medium 722 can be arandom-access memory or other volatile re-writable memory. Additionallyor alternatively, the computer-readable medium 722 can include amagneto-optical or optical medium, such as a disk or tapes or otherstorage device to capture carrier wave signals such as a signalcommunicated over a transmission medium. A digital file attachment to ane-mail or other self-contained information archive or set of archivesmay be considered a distribution medium that is a tangible storagemedium. Accordingly, the disclosure is considered to include any one ormore of a computer-readable medium or a distribution medium and otherequivalents and successor media, in which data or instructions may bestored.

In an alternative implementation, dedicated hardware implementations,such as application specific integrated circuits, programmable logicarrays and other hardware devices, can be constructed to implement oneor more of the methods described herein. Applications that may includethe apparatus and systems of various implementations can broadly includea variety of electronic and computer systems. One or moreimplementations described herein may implement functions using two ormore specific interconnected hardware modules or devices with relatedcontrol and data signals that can be communicated between and throughthe modules, or as portions of an application-specific integratedcircuit. Accordingly, the present system encompasses software, firmware,and hardware implementations.

The computer system 700 may be connected to a network 770. The network770 may define one or more networks including wired or wirelessnetworks. The wireless network may be a cellular telephone network, an802.11, 802.16, 802.20, or WiMAX network. Further, such networks mayinclude a public network, such as the Internet, a private network, suchas an intranet, or combinations thereof, and may utilize a variety ofnetworking protocols now available or later developed including, but notlimited to TCP/IP based networking protocols. The network 770 mayinclude wide area networks (WAN), such as the Internet, local areanetworks (LAN), campus area networks, metropolitan area networks, adirect connection such as through a Universal Serial Bus (USB) port, orany other networks that may allow for data communication. The network770 may be configured to couple one computing device to anothercomputing device to enable communication of data between the devices.The network 770 may generally be enabled to employ any form ofmachine-readable media for communicating information from one device toanother. The network 770 may include communication methods by whichinformation may travel between computing devices. The network 770 may bedivided into sub-networks. The sub-networks may allow access to all ofthe other components connected thereto or the sub-networks may restrictaccess between the components. The network 770 may be regarded as apublic or private network connection and may include, for example, avirtual private network or an encryption or other security mechanismemployed over the public Internet, or the like.

In accordance with various implementations of the present disclosure,the methods described herein may be implemented by software programsexecutable by a computer system. Further, in an exemplary, non-limitedimplementation, implementations can include distributed processing,component/object distributed processing, and parallel processing.Alternatively, virtual computer system processing can be constructed toimplement one or more of the methods or functionality as describedherein.

Although the present invention has been described above and is definedin the attached claims, it should be understood that the invention mayalternatively be defined in accordance with the following embodiments:

-   1. A computer-implemented method (100) for    handwriting-to-text-summarization, comprising:    -   obtaining (110), via a user interface (210) of a system (200), a        handwriting input (10) representing a handwriting of a user of        the system (200) for handwriting-to-text-summarization;    -   recognizing (120) a text (20) in the handwriting input (10);    -   extracting (130) at least one dynamic feature (30) of the        handwriting from the handwriting input (10);    -   generating (140) a text summary (40) of the text (20);    -   wherein generating (140) the text summary (40) is based on the        text (20) and on the at least one dynamic feature (30) of the        handwriting.-   2. The method (100) of embodiment 1, wherein the one or more dynamic    features (30) comprise an average writing pressure, an average    stroke length, and/or an average stroke duration, wherein averaging    is over the text (20) or portions thereof.-   3. The method (100) of embodiment 1 or 2, wherein the text (20) is    semantically and/or linguistically interpretable with respect to at    least one communication language.-   4. The method (100) of one of the preceding embodiments, wherein the    handwriting input (10) comprises a first set of data representing    the text.-   5. The method (100) of one of the preceding embodiments, wherein the    handwriting input (10) comprises a second set of data representing    properties of the handwriting that indicate information about the    user as handwriting progresses.-   6. The method (100) of embodiment 5, wherein the information about    the user relates to an emotional state, an age, and/or a level of    competence of the user.-   7. The method (100) of embodiment 5 or 6, when dependent on    embodiment 4, wherein the first set of data and the second set of    data overlap or are identical.-   8. The method (100) of one of the preceding embodiments, wherein    recognizing (120) the text (20) in the handwriting input (10)    comprises applying the handwriting input (10) to a text    pre-processing algorithm configured to recognize the text (20)    represented by the handwriting input (10).-   9. The method (100) of embodiment 8, wherein the text pre-processing    algorithm is further configured to segment (121) the text (20) into    one or more portions.-   10. The method (100) of embodiment 9, wherein the one or more    portions are sentences, clauses, and/or phrases.-   11. The method (100) of embodiment 9 or 10, wherein the one or more    portions are enumerated as they appear in the text.-   12. The method (100) of one of the embodiments 8 to 11, wherein the    text pre-processing algorithm comprises or is a machine-learning    algorithm pre-trained for handwriting-to-text recognition and/or    text segmentation.-   13. The method (100) of one of the embodiments 8 to 12, when    dependent on embodiment 4, wherein the text pre-processing algorithm    is configured to segment the first set of data into individual    character vectors and apply a pre-determined vector-to-character    mapping to output the text.-   14. The method (100) of one of the preceding embodiments, wherein    the text pre-processing algorithm is further configured to store the    text (20) in a database.-   15. The method (100) of one of the preceding embodiments, when    dependent on embodiment 9, wherein the text pre-processing algorithm    is further configured to store the one or more portions of text (20)    in a database.-   16. The method (100) of one of the preceding embodiments, wherein    extracting (130) the at least one dynamic feature (30) of the    handwriting from the handwriting input (10) comprises applying the    handwriting input (10) to a handwriting dynamics algorithm    configured to extract the at least one dynamic feature (30) from the    handwriting input (10).-   17. The method (100) of embodiment 16, wherein the handwriting    dynamics algorithm is further configured to compute at least one    dynamic feature (30) for each portion of the text.-   18. The method (100) of embodiment 16 or 17, when dependent on    embodiment 2, wherein the handwriting dynamics algorithm is further    configured to compute the average writing pressure, the average    stroke length, and/or the average stroke duration, wherein averaging    is over the text (20) or portions thereof, thereby producing the one    or more dynamic features (30) for the text (20) or for each portion    thereof.-   19. The method (100) of embodiment 18, when dependent on embodiment    5, wherein computing the average writing pressure, the average    stroke length, and/or the average stroke duration is based on the    second set of data.-   20. The method (100) of one of the preceding embodiments, when    dependent on embodiment 14, wherein the handwriting dynamics    algorithm is further configured to store the one or more dynamic    features (30) for the text (20) in the database.-   21. The method (100) of one of the preceding embodiments, when    dependent on embodiment 15, wherein the handwriting dynamics    algorithm is further configured to store the one or more dynamic    features (30) for each portion of the text (20) in the database.-   22. The method (100) of one of the preceding embodiments, wherein    generating (140) the text summary (40) of the text (20) based on the    text (20) and on the at least one dynamic feature (30) of the    handwriting comprises applying the text (20) and the one or more    dynamic features (30) to a text summarization algorithm configured    to generate the text summary (40) of the text (20).-   23. The method (100) of embodiment 22, wherein the text    summarization algorithm comprises or is a natural language    processing machine-learning algorithm pre-trained for    extraction-based or abstraction-based text summarization.-   24. The method (100) of embodiment 22 or 23, when dependent on    embodiment 9, wherein the text summarization algorithm comprises    applying (141) the one or more portions of the text (20) to an    importance classifier configured to classify each portion of the    text (20) in terms of at least two classes indicating different    levels of importance.-   25. The method (100) of embodiment 24, wherein the importance    classifier is a natural language processing machine-learning    algorithm pre-trained for extraction-based text summarization.-   26. The method (100) of embodiment 24 or 25, wherein the text    summarization algorithm comprises applying (142) the one or more    dynamic features (30) for each portion of the text (20) to at least    one quality classifier configured to classify the one or more    dynamic features (30) for each portion of the text (20) in terms of    at least two classes relating to information about the user.-   27. The method (100) of embodiment 26, when dependent on embodiment    6, wherein the at least one quality classifier corresponds to one or    more of the emotional state, the age, and the level of competence of    the user.-   28. The method (100) of embodiment 26 or 27, wherein the at least    one quality classifier corresponds to the level of competence of the    user.-   29. The method (100) of embodiment 28, wherein classifying the level    of competence of the user is in terms of two or three classes.-   30. The method (100) of one of the embodiments 26 to 29, wherein the    one or more quality classifiers are machine-learning algorithm in a    pre-trained state.-   31. The method (100) of one of the preceding embodiments, when    dependent on embodiment 26, wherein the text summarization algorithm    comprises applying (143) a portion ranking algorithm configured to    assign a ranking value to each portion of the text (20) based on the    corresponding classification results (41) of the importance    classifier and on the corresponding classification results (42) of    the at least one quality classifier.-   32. The method (100) of embodiment 31, wherein assigning a ranking    value to each portion of the text (20) based on the corresponding    classification results (41) of the importance classifier and on the    corresponding classification results (42) of the at least one    quality classifier comprises computing a linear combination of a    numeric result (IC) of the importance classifier and a normalized    linear combination of numeric results (QC₁, QC₂, . . . , QC_(N)) of    the N quality classifiers;    -   wherein the numeric results of the importance classifier and of        the N quality classifiers are pre-determined values        corresponding to the respective classes of the importance        classifier or the N quality classifiers, respectively; and    -   wherein coefficients of the linear combination and/or        coefficients of the normalized linear combination are        pre-determined weights.-   33. The method (100) of embodiment 31 or 32, wherein the portions of    the text (20) with corresponding ranking values above a    predetermined threshold value are concatenated, thereby generating    (140) the text summary (40).-   34. The method (100) of one of the preceding embodiments, when    dependent on embodiment 5, further comprising applying (150) the    text summary (40) to a style transfer algorithm configured to modify    the text summary (40) so as to reflect the information about the    user.-   35. The method (100) of embodiment 34, wherein the style transfer    algorithm applies at least one style transfer network.-   36. The method (100) of embodiment 34 or 35, when dependent on    embodiments 9 and 26, wherein modifying the text summary (40)    comprises:    -   selecting for each portion of the text summary (40) one or more        style transfer networks based on the one or more classification        results (42) of the one or more quality classifiers; and    -   modifying each portion of the text summary (40) by applying the        corresponding one or more transfer networks;

thereby modifying the text summary (40).

-   37. The method (100) of embodiment 35 or 36, wherein the at least    one style transfer network is an auto-encoder neural network in a    pre-trained state.-   38. The method (100) of one of the preceding embodiments, when    dependent on embodiments 5 and 9, further comprising applying (160)    the text summary (40) to a font modification algorithm configured to    change the font of at least one portion of the text summary (40)    based on the information about the user.-   39. The method (100) of embodiment 38, wherein changing the font of    the at least one portion of the text summary (40) based on the    information about the user comprises:    -   querying for each portion of the text summary (40) a font based        on the one or more classification results (42) of the one or        more quality classifiers matching the font labels of the font;    -   formatting each portion in the corresponding font;

thereby modifying the text summary (40).

-   40. The method (100) of embodiment 38 or 39, wherein at least one    font is queried from a font database.-   41. The method (100) of one of the preceding embodiments, when    dependent on embodiment 5, further comprising applying (170) the    text summary (40) to an annotation algorithm configured to add at    least one annotation to the text summary (40) based on the    information about the user.-   42. The method (100) of embodiment 41, when dependent on embodiments    9 and 26, wherein adding the at least one annotation to the text    summary (40) based on the information about the user comprises:    -   querying, for each portion of the text summary (40), an        annotation database so as to find a matching annotation to the        portion and to the one or more classification results (42) of        the one or more quality classifiers; and    -   evaluating a predetermined trigger condition; and    -   adding the matching annotation to the portion, if a matching        annotation has been found and the predetermined trigger        condition is satisfied; or    -   adding a universal annotation to the portion, if a matching        annotation has not been found and the predetermined trigger        condition is satisfied;

thereby modifying the text summary (40).

-   43. The method (100) of embodiment 42, wherein the matching    annotation or the universal annotation results from a queried    annotation template after parametrization based on the portion    and/or the one or more classification results (42) of the one or    more quality classifiers.-   44. The method (100) of embodiment 43, wherein the universal    annotation is an interactive hyperlink to a search engine.-   45. The method (100) of one of the preceding embodiments, when    dependent on embodiment 5, further comprising applying (180) the    text summary (40) to a collation algorithm configured:    -   to compare the text summary (40) to one or more further text        summaries corresponding to one or more further users of one or        more further systems (200) for        handwriting-to-text-summarization; and    -   to sample a collated text summary (40) based on the information        about the user and on information about the one or more further        users;

thereby modifying the text summary (40).

-   46. The method (100) of embodiment 45, when dependent on embodiment    9, wherein comparing the text summary (40) to one or more further    text summaries comprises identifying for each portion of the text    summary (40) a list of corresponding further portions in the one or    more further text summaries.-   47. The method (100) of embodiment 46, wherein sampling a collated    text summary (40) based on the information about the user and on the    information about the one or more further users comprises selecting    a portion or a further portion for each list based on the    information about the user and on the information about the one or    more further users.-   48. The method (100) of embodiment 47, wherein the information about    the user and the information about the one or more further users    correspond to the level of competence of the user or the further    users, respectively.-   49. The method (100) of one of the preceding embodiments, wherein    user settings can be set via the user interface (210) of the system    (200).-   50. The method (100) of one of the preceding embodiments, further    comprising displaying (190) the text summary (40) via a graphical    output (230) of the user interface (210) of the system (200) for    handwriting-to-text-summarization.-   51. A system (200) for handwriting-to-text-summarization, comprising    a user interface (210) comprising a capturing subsystem (220)    configured to capture a handwriting of a user of the system (200),    and wherein the system (200) is configured to run the method (100)    of one of the preceding embodiments.-   52. The system (200) of embodiment 51, wherein the capturing    subsystem (220) comprises a touchpad, a touch screen, a graphics    tablet, or a digital tablet.-   53. The system (200) of embodiment 52, wherein capturing the    handwriting of the user of the system (200) comprises capturing    handwriting input (10) representing text (20) written by hand or    writing utensil (221) by the user on the touchpad, the touch screen,    the graphics tablet, or the digital tablet.-   54. The system (200) of one of the embodiments 51 to 53, wherein the    capturing subsystem (220) comprises or is integrated in a writing    utensil (221).-   55. The system (200) of embodiment 54, wherein the writing utensil    (221) is a ballpoint pen, a fountain pen, a felt-tip pen, a brush, a    pencil.-   56. The system (200) of embodiment 54, wherein the writing utensil    (221) is a digital pen or a smart pen.-   57. The system (200) of embodiment 55 or 56, wherein the writing    utensil (221) comprises a motion sensor module (222) comprising one    or more of one or more accelerometers, one or more gyroscopes, one    or more magnetometers, and one or more force or pressure sensors.-   58. The system (200) of embodiment 57, wherein the motion sensor    module (222) is configured to capture handwriting motion data    comprising one or more of pressure, speed, acceleration, and    direction of the writing utensil (221) when being used by the user    to write text (20) by hand.-   59. The system (200) of embodiment 57 or 58, wherein capturing the    handwriting of the user of the system (200) comprises capturing    handwriting input (10) comprising handwriting motion data of the    writing utensil (221) when being used by the user to write text (20)    by hand.-   60. The system (200) of one of the embodiments 51 to 59, wherein the    handwriting input (10) comprises a first set of data representing    the text.-   61. The system (200) of one of the embodiments 51 to 60, wherein the    handwriting input (10) comprises a second set of data representing    properties of the handwriting that indicate information about the    user as handwriting progresses.-   62. The system (200) of embodiment 60 or 61, wherein the first set    of data and the second set of data overlap or are identical.-   63. The system (200) of one of the embodiments 51 to 62, wherein the    user interface (210) comprises a graphical output (230).-   64. The system (200) of embodiment 63, wherein the graphical output    (230) allows for user interaction.-   65. The system (200) of one of the embodiments 51 to 64, comprising    at least one database.-   66. The system (200) of one of the embodiments 51 to 65, comprising    a communication interface (240) to couple to one or more systems    (200) for handwriting-to-text-summarization.

REFERENCE NUMERALS

-   10 handwriting input-   20 text (in the handwriting input)-   30 at least one dynamic feature-   40 text summary (of the text)-   41 classification result of the importance classifier-   42 classification result(s) of the at least one quality classifier-   100 computer-implemented method for    handwriting-to-text-summarization-   110 obtaining a handwriting input representing a handwriting of a    user of the system for handwriting-to-text-summarization-   120 recognizing a text in the handwriting input-   121 segment the text into one or more portions-   130 extracting at least one dynamic feature of the handwriting from    the handwriting input-   140 generating a text summary of the text-   141 applying the one or more portions of the text to an importance    classifier-   142 applying the one or more dynamic features for each portion of    the text to at least one quality classifier-   143 applying a portion ranking algorithm-   150 applying the text summary to a style transfer algorithm-   160 applying the text summary to a font modification algorithm-   170 applying the text summary to an annotation algorithm-   180 applying the text summary to a collation algorithm-   190 displaying the text summary via a graphical output of the user    interface of the system for handwriting-to-text-summarization-   200 system for handwriting-to-text-summarization-   210 user interface-   220 capturing subsystem-   221 writing utensil-   222 motion sensor module-   230 graphical output-   240 communication interface

1. A computer-implemented method for handwriting-to-text-summarization,comprising: obtaining, via a user interface of a system, a handwritinginput representing a handwriting of a user of the system for thehandwriting-to-text-summarization; recognizing a text in the handwritinginput; extracting at least one dynamic feature of the handwriting fromthe handwriting input; and generating a text summary of the text,wherein generating the text summary is based on the text and on the atleast one dynamic feature of the handwriting.
 2. Thecomputer-implemented method of claim 1, wherein the at least one dynamicfeature comprises an average writing pressure, an average stroke length,an average stroke duration, or a combination thereof, and whereinaveraging is over the text or portions thereof.
 3. Thecomputer-implemented method of claim 1, wherein the handwriting inputcomprises a first set of data representing the text.
 4. Thecomputer-implemented method of claim 1, wherein the handwriting inputcomprises a second set of data representing properties of thehandwriting that indicate information about the user as the handwritingprogresses.
 5. The computer-implemented method of claim 3, whereinrecognizing the text in the handwriting input comprises applying thehandwriting input to a text pre-processing algorithm configured torecognize the text represented by the handwriting input.
 6. Thecomputer-implemented method of claim 5, wherein the text pre-processingalgorithm is further configured to segment the text into one or moreportions.
 7. The computer-implemented method of claim 5, wherein thetext pre-processing algorithm comprises or is a machine-learningalgorithm pre-trained for handwriting-to-text recognition and/or textsegmentation.
 8. The computer-implemented method of claim 5, wherein thetext pre-processing algorithm is configured to segment the first set ofdata into individual character vectors and apply a pre-determinedvector-to-character mapping to output the text.
 9. Thecomputer-implemented method of claim 2, wherein extracting the at leastone dynamic feature of the handwriting from the handwriting inputcomprises applying the handwriting input to a handwriting dynamicsalgorithm configured to extract the at least one dynamic feature fromthe handwriting input.
 10. The computer-implemented method of claim 9,wherein the handwriting dynamics algorithm is further configured tocompute one or more of the average writing pressure, the average strokelength, the average stroke duration, or a combination thereof, whereinthe averaging is over the text or portions thereof, thereby producingthe at least one dynamic feature for the text or for each portionthereof.
 11. The computer-implemented method of claim 6, whereingenerating the text summary of the text based on the text and on the atleast one dynamic feature of the handwriting comprises applying the textand the at least one dynamic feature to a text summarization algorithmconfigured to generate the text summary of the text.
 12. Thecomputer-implemented method of claim 11, wherein the text summarizationalgorithm comprises applying the one or more portions of the text to animportance classifier configured to classify each portion of the text interms of at least two classes indicating different levels of importance.13. The computer-implemented method of claim 4, further comprisingapplying the text summary to a collation algorithm configured for:comparing the text summary to one or more further text summariescorresponding to one or more further users of one or more furthersystems for the handwriting-to-text-summarization; and sampling acollated text summary based on the information about the user and oninformation about the one or more further users, thereby modifying thetext summary.
 14. The method of claim 13, further comprising: applyingthe text summary to a style transfer algorithm configured to modify thetext summary so as to reflect the information about the user.
 15. Themethod of claim 13, further comprising: applying the text summary to afont modification algorithm configured to change the font of at leastone portion of the text summary based on the information about the user.16. A system for handwriting-to-text-summarization, comprising a userinterface comprising a capturing subsystem configured to capture ahandwriting of a user of the system, wherein the system is configuredfor: obtaining, via the user interface of the system, a handwritinginput representing the handwriting of the user of the system for thehandwriting-to-text-summarization; recognizing a text in the handwritinginput; extracting at least one dynamic feature of the handwriting fromthe handwriting input; and generating a text summary of the text,wherein generating the text summary is based on the text and on the atleast one dynamic feature of the handwriting.
 17. The system of claim16, further comprising: a communication interface to couple to one ormore systems for the handwriting-to-text-summarization.
 18. The systemof claim 16, wherein the at least one dynamic feature comprises anaverage writing pressure, an average stroke length, an average strokeduration, or a combination thereof, and wherein averaging is over thetext or portions thereof.
 19. The computer-implemented method of claim1, wherein the handwriting input comprises a first set of datarepresenting the text.
 20. A method forhandwriting-to-text-summarization, comprising: obtaining, via a userinterface of a system, a handwriting input representing a handwriting ofa user of the system for the handwriting-to-text-summarization;recognizing a text in the handwriting input, wherein the text issemantically and/or linguistically interpretable with respect to atleast one communication language; extracting at least one dynamicfeature of the handwriting from the handwriting input, wherein thehandwriting input comprises a first set of data representing the text;and generating a text summary of the text, wherein generating the textsummary is based on the text and on the at least one dynamic feature ofthe handwriting.